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Met:hod and Appara-bus for performing synchronised audio and 
video presexi-ba-tlon 

The invention relates to the synchronised presentation or 
5 reproduction of video and audio streams using non-synchro- 
nised processing means. 

Background 

10 

MPEG-4 is an international standard developed by the Motion 
Picture Experts Group (MPEG) which also developed a number 
of other MPEG-type standards for compressing audio and video 
data^ for example MPEG-1 and MPEG-2 . The encoded/compressed 

15 data is treated as object data and both,, video and audio 

data, are combined into a single bitstream. Since an MPEG-4 
system is configured to treat data as object data, it is 
easy to re-organise a received bitstream by separating it 
into multiple single packets of data. An MPEG-4 player al- 

20 lows then the audio and video data to be reproduced on a 
computer or an other device. 

Invention 

25 

Even though the video encoding associated with the MPEG-type 
standard provides high resolution pictures, its use requires 
one or more powerful, dedicated processors, for example a 
digital signal processor, for encoding or decoding MPEG-type 

30 standard video data. The processing of an entire MPEG-type 
stream using only one computer consumes nearly all of the 
computational resources of the computer's general purpose 
CPU (central processing unit) , thereby rendering the com- 
puter virtually useless .for any other purpose. As a conse- 

35 quence, being able to use for MPEG-type technology a network 
of remote computers or devices and processing a video stream 
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on one computer or device while sending the audio data to be 
processed on a second computer or device is highly desirable 
for MPEG-type data processing since it allows the utilisa- 
tion of any standard computer or device for the video and 
5 audio processing. 

Thus, it would be desirable to use two processing means or 
computers for rendering or presenting video and audio data, 
wherein video and audio streams need to be synchronised for 
presentation or reproduction. 

10 

A problem to be solved by the invention is to provide syn- 
chronised presentation or reproduction of video and aud±o 
using separate devices the operation of which is basically 
not synchronised with each other. This problem is solved by 
15 the method disclosed in claim 1. An apparatus that util±ses 
this method is disclosed in claim 8. 

Advantageous additional embodiments of the invention are 
disclosed in the respective dependent claims. 

20 The inventive features described below are used for synohro- 
nising presentation or reproduction of audio data with the 
appropriate video data utilising two processing means ojo 
computers. A data stream comprising video and audio streams 
is received by first processing means, the received data. 

25 stream is separated into video and audio streams and audio 

stream audio data packets are timestamped by the first proc- 
essing means. Then, audio data packets are forwarded to sec- 
ond processing means, a local system time of the second 
processing means is determined and transmission time peiriods 

30 of audio data packets from the first processing means to the 
second processing means are calculated based on the local 
system time and the timestamp of the audio data packet. Sub- 
sequently, synchronised audio and video rendering/presen- 
tation based on the transmission time period is performed. 

35 Advantageously the process of rendering is accompanied toy 
lowpass filtering the transmission time periods whereby a 
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mean transmission time is obtained and used for synchronisa- 
tion of video and audio presentation or reproduction. Also a 
median filter can be used for lowpass filtering the measured 
transmission time periods in order to the measurement re- 
5 suit- 

The present invention solves the above-mentioned problems of 
the prior art^ and provides a method capable of fast re- 
sponse at start-up^ as well as high stability during proc- 
essing. The median filter is also very insusceptible with 

10 respect to large measuring errors. 

An MPEG-type stream is separated into video data and audio 
data, wherein the video data is processed on the first de- 
vice PC_A and the audio data is timestamped and forwarded to 
the second device PC_B which compares the received timestamp 

15 to the local time. The difference is considered to be the 

required transmission time. The internal time clocks of the 
first processing device and the second processing device are 
not synchronised. 

The time reference for synchronisation of video and audio 
20 stream is obtained by subtracting mean transmission time pe- 
riods from the local time of the second processing device 
PC_B. Subsequently, an additional lowpass filtering can be 
performed by a digital filter such as a Butterworth filter 
having a cut-off frequency below that of high frequency mo- 
25 tion (jitter) which needs to be eliminated. 



Drawings 

30 Exemplary embodiments of the invention are described with 
reference to the accompanying drawings, which show in: 
Fig. 1 Block diagram illustrating a network of first and 

second processing means configured to perform audio 
and video presentation or reproduction; 
35 Fig. 2 Flowchart of the inventive process. 
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Exemplary embodiments 



In the block diagram of Fig. 1 showing a general configura- 
tion of a multimedia computer network according to the in- 
5 vention, the reference numeral 100 denotes an MPEG-4 player 
which sends an MPEG-4 data stream 102 to first processing 
means PC_A 104 which include a video player 108. The re- 
ceived MPEG-type stream comprises a system^ video and audio 
stream, which further contain video data packets 116 and au- 

10 dio data packets 134. 

A stream analysing stage 110 examines the streams since the 
system stream also includes the structure and the configura- 
tion of the video and audio players. The first computer PC_A 
104 processes video data obtained from the MPEG-4 video 

15 stream and displays it using e.g. an attached monitor. The 
timestamping stage 112 checks the local time clock 106 and 
inserts timestamps into audio data packets. A network 118 
e.g. of type Ethernet (TCP/IP);, connects the first process- 
ing means 104 with second processing means 120;. ^-g- a sec- 

20 ond computer PC_B, which processes the audio data packets 
received from the first computer PC__A, using audio player 
126. The time base 114 of the first computer 104 and the 
time base 132 of the second computer 120 are not synchro- 
nised with each other and they have a tendency to drift away 

25 from each other. The second computer or the network or the 
first computer checks the local time clock 122 and compares 
the received timestamp 124 to the local time of time clock 
122. The second computer or the network or the first com- 
puter calculates the corresponding transmission time peri- 

30 ods. 

A median filter 128 can be used for lowpass filtering of 
transmission time periods in order to obtain mean transmis- 
sion which is in turn used for synchronisation of audio and 
video rendering. A Butterworth filter 130 provides addi- 
35 tional lowpass filtering in order to improve the final re- 
sult. 
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MPEG-4 player 100 sends the MPEG-4 stream of data to the 
first processing means PC_A which processes video data and 
also forwards the actualised and timestamped audio data 
packets to the second computer PC_B through the network. Af- 
5 ter receiving audio data packets and also its configuration 
from the first computer^ the second computer PC_B compares 
the received timestamp to the local time. The difference is 
considered to be the transmission time period* 
The time base of the video processing computer 104 is not 

10 synchronised with the time base of the audio processing com- 
puter 120- Also, the internal time clocks of the first and 
the second computer are not synchronised and slowly drift 
from each other. Thus, the timestamps received by the second 
computer can be considered as being altered with respect to 

15 their value because the real transmission time cannot be 

specified exactly. This may have different reasons, for ex- 
ample: traffic on the network line or lines, configuration 
of TCP/IP and Ethernet, thread processing of the operating 
system, the varying amount of data, etc.. In order to syn- 

20 chronise the presentation or reproduction of audio data with 
the appropriate video data the time difference between the 
sending of the packets and their receiving is calculated. 
This difference is then filtered with a median filter. 
A median filter is a time-discrete, non-linear filter which 

25 stores the acquired samples, sorts them and provides the 

middle sample (or the average of the two middle samples in 
case of even number of input values) as an output of its op- 
eration- The median filter used for the invention is very 
flexible with respect to the number of input samples it 

30 processes. Initially all samples values are set to zero. Af- 
ter having collected a pre-defined first number of samples, 
e.g. 19, the median filter starts outputting the mean trans- 
mission time, whereby the length of the median filter corre- 
sponds to said first number. As an option, upon receiving 

35 further input samples, the filter length used is increased 
by one per additional input sample received, up to a pre- 
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defined maximiim length, e.g. 4 99. Thereby both, a fast reac- 
tion time at start-up and a stable continuous operation can 
be achieved. Subsequently, an additional lowpass filtering 
can be performed by a digital filter such as a Butterworth 
5 filter having a cut-off frequency below that of high fre- 
quency motion (jitter) which needs to be eliminated. This 
kind of operation allows synchronising the video and audio 
presentation with respect to time and thus eliminating dis- 
continuities in the time bases of the two computers. 

10 

In the step 200 in the flow chart of Fig. 2, an. MPEG-type 
stream comprising video and audio streams is received by the 
first processing means PC_A. In the next step 202 the said 
MPEG-type data stream is separated into the video and the 

15 audio streams, wherein the first processing computer PC_A 
containing video player processes the video stream and the 
second processing computer PC_B containing the audio player 
processes the audio stream. Subsequently, in the step 204, 
the audio data packets are timestamped by the video process- 

20 ing computer PC_A and forwarded to the audio processing com- 
puter PC_B configured to receive audio data from the video 
processing computer PC_A. In the next step 20 6, the local 
system time of the audio processing computer is determined. 
Next, the audio stream transmission time periods from the 

25 first processing means to the second processing means are 
calculated in step 208. In the last step 210 synchronising 
of audio and video presentation or reproduction based on the 
calculated transmission time periods takes place. 



30 Instead of synchronising presentation or reproduction of au- 
dio data with the appropriate video data, the presentation 
or reproduction of video data with the appropriate audio 
data can be performed. In such case video data packets of 
the video stream are timestamped by the first processing 

35 means and video data packets are forwarded to the second 
processing means configured to receive audio data packets. 
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Time periods are calculated for the transmission of the 
video data packets from the first processing means to the 
second processing means ^ based on the corresponding local 
system time and the timestamps of the video data packets - 

5 



